AI Tool Use Among Osteopathic Medical Students: Pilot Digital Diary Study

Introduction

The integration of AI into medical education is rapidly transforming the way students acquire knowledge, develop clinical reasoning skills, and engage in lifelong learning [1]. In higher education, AI-powered tools, such as machine learning algorithms, large language models, and automated tutoring programs, are increasingly being used to enhance traditional learning methods [2].

Within medical education specifically, AI tools are being used to support self-directed learning, concept review, and examination preparation. ChatGPT has been reported to perform at or above the median performance among 276,779 student test takers on the Medical College Admission Test (MCAT) and has performed at or near the passing threshold for all 3 steps of the United States Medical Licensing Examination (USMLE) [3,4]. These capabilities have driven rapid student adoption, with surveys suggesting between 52% and 90% of medical students in various settings have used AI tools for studying [5-8]. AI is also influencing professional development, with 43% of residency applicants planning to use AI for personal statements [9].

The ethical discussions surrounding AI use by medical students are still developing, emphasizing the importance of understanding its limitations, verifying sources, ensuring Health Insurance Portability and Accountability Act (HIPAA) compliance, and maintaining academic integrity [10]. Critically, there is limited guidance specific to student use of AI during training, as distinct from clinical or research contexts [11,12].

Existing evidence suggests that medical students have a range of knowledge of AI tools. A 2023 systematic review found that students had generally positive attitudes toward AI in medicine, but most had low knowledge and limited skills [13]. By 2024, high knowledge and high usage (70%) were reported among 102 US medical students, with AI exposure associated with greater trust in clinical AI applications [14].

However, how students engage with large language models during studying remains incompletely characterized. A survey of 415 students from 28 US medical schools found 52% used AI tools for schoolwork, primarily for concept explanation during preclerkship and diagnostic assistance during clerkship [15]. While informative, these studies relied on retrospective recall, did not quantify the proportion of study time devoted to AI, and did not capture real-time patterns of use. To our knowledge, no prior study has used a prospective diary methodology to measure AI use as a percentage of total study time.

Digital diaries, also called ecological momentary assessment or experience sampling, are a validated method for prospective data collection that reduce recall bias by capturing experiences in or near real time [16,17]. Unlike learning portfolios, which are reflective and often summative, digital diaries capture ongoing behaviors at regular intervals with minimal retrospective reconstruction.

This study assessed the adoption and use of AI tools by medical students during studying and schoolwork through a longitudinal digital diary to capture accurate accounts of the nature and frequency of AI use. These findings may inform institutional policies, curriculum design, and preparation of future health care professionals to navigate AI in clinical practice.

Methods

Study Design and Setting

This was a longitudinal observational cohort study using a digital diary methodology. The study was conducted at Touro University California (TUC) and Touro University Nevada (TUN), both accredited colleges of osteopathic medicine in the United States. This study is reported in accordance with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement for cohort studies; the completed checklist is provided in Checklist 1. Recruitment spanned from April 10 to 28, 2025. Each participant began a 21-day digital diary upon completing the baseline survey; the final diary entry was collected on May 19, 2025. The study consisted of a one-time intake survey followed by a 21-day digital diary phase (Figure 1). Digital diaries have been shown to be a valid method of collecting responses while minimizing recall bias [16]. Participants received survey prompts 3 times per week for 3 weeks—an approach that balanced reducing recall bias and maximizing participation [17].

‎

No formal power calculation was conducted; the sample represents a convenience sample of all willing enrollees during the recruitment period, consistent with the pilot or exploratory aims of the study.

Participants and Recruitment

All currently enrolled doctor of osteopathy (DO) students from both campuses were eligible to participate. Recruitment was conducted through email announcements distributed via institutional listservs. A total of 1332 students were invited to participate: 601 via the TUC listserv and 731 via the TUN listserv. Interested students received an electronic informed consent form and were enrolled in the study upon providing consent. Of those invited, 71 (5.3%) students expressed interest, consented, and were enrolled; there were no withdrawals and 7 losses to follow-up during the 21-day diary period. The study recruitment period spanned from April 10 to 28, 2025. Each participant was on their own 21-day digital diary schedule beginning upon the completion of their intake survey. The final digital diary entry was collected on May 19, 2025.

To assess representativeness, we compared our sample’s gender composition with DO program-specific institutional enrollment data. Women comprise approximately 54% of DO students at TUC and 47% at TUN. Weighting these benchmarks by our sample’s campus composition (76% TUC, 24% TUN) yields an expected population value of 52.3% women, compared with 45.1% (32/71) among enrolled participants; this difference was not statistically significant (z=−1.22; P=.22)

Data Collection

Following informed consent, participants completed the baseline intake survey via Google Forms. The survey collected demographic information (age, self-reported gender [options: man, woman, nonbinary, other, and prefer not to say], year in training, campus), prior exposure to AI tools, perceived impact of AI on learning and clinical reasoning, and self-reported frequency and use cases of AI tools. The survey is available as a supplementary file (Multimedia Appendix 1).

After completing the intake survey, participants received a brief digital survey prompt every 3 days over a 21-day period (7 total entries). Surveys were delivered via automated SMS using SimpleTexting [18]. Each diary asked students to report (1) total minutes spent studying in the past 72 hours, (2) minutes using AI tools, (3) which tools were used, and (4) purposes of use (eg, concept explanation, summarization, question generation). “Total study time” was defined as self-reported minutes spent on academic activities (reading, reviewing notes, practice questions, school-related tasks) during the prior 72 hours. All responses were stored in a secure, password-protected cloud folder.

Data Analysis

The data were analyzed using Stata 19 [19]. Descriptive statistics summarized all baseline variables. Digital diary data were aggregated to generate individual-level scores representing frequency and type of AI use. Subgroup analyses examined differences by year in training (preclinical vs clinical), gender, age, campus, and active versus passive use. Seven students who completed the intake survey did not submit any diary entries and were excluded from diary-based analyses (final analytic sample n=64). Completers and noncompleters did not differ significantly on measured demographic characteristics, suggesting exclusion is unlikely to have introduced systematic bias.

Active and passive use were defined a priori based on the cognitive engagement literature, drawing on emerging conceptual work on active versus passive AI engagement [20,21]. Active use was defined as iterative, bidirectional engagement with AI tools requiring critical appraisal and integration of outputs (eg, iterative prompting for clinical reasoning, evaluating competing diagnoses). Passive use was defined as unidirectional consultation with limited interrogation of outputs (eg, requesting a summary or explanation without follow-up).

Bivariate analyses (2-tailed t tests, chi-square tests) assessed group differences. Multiple linear regression examined the relationship between AI use (percentage of study time; total minutes) and key predictors, adjusting for age, gender, year in training, and active or passive use. Effect sizes for bivariate comparisons were calculated as Cohen d (small: d≥0.20, medium: d≥0.50, large: d≥0.80).

Regression analyses were conducted using listwise deletion. Effect sizes for regression predictors were calculated as Cohen f², computed as the difference in R² between the full model and a reduced model excluding each predictor, divided by 1 minus the full model R² (small: f²≥0.02, medium: f²≥0.15, large: f²≥0.35).

To directly address the repeated-measures structure of the diary data, we conducted mixed-effects (multilevel) regression models using diary-level rather than person-aggregated data, with a random intercept for each student to account for the nonindependence of repeated diary entries.

Ethical Considerations

This study was approved by the TUC Institutional Review Board (IRB; TUC IRB application M-1425). All participants provided written informed consent. The study adhered to the Declaration of Helsinki. All research team members completed Collaborative Institutional Training Initiative (CITI) training. No identifying information was collected, and data were stored securely in compliance with federal data protection standards. As an incentive, students who completed the study were entered in a raffle for twenty-eight US $25 Visa gift cards and four US $50 Target gift cards.

Results

Participant Characteristics

Of 1332 students invited via institutional listservs (601 at TUC; 731 at TUN), 71 (5.3%) consented and enrolled. No participants withdrew, while 7 did not complete any diaries. A total of 71 medical students completed the intake survey (Figure 2). The average age was 26.6 (SD 2.8) years. Among respondents, 39 (55%) identified as men and 32 (45%) identified as women. Students from all class years participated: 55 (77%) were in preclinical years MS1 or MS2, and 16 (23%) were in clinical years MS3, MS4, or research year. A total of 54 students (76%) were from TUC and 17 (24%) were from TUN. Of these, 32 completed all 7 diary entries, 26 completed 4 to 6 entries, and 7 completed 1 to 3 entries, and 7 completed 0 entries. Total diary entries received were as follows: 381 of 497 possible (71×7), representing 76.7% completion.

‎

Regarding prior AI exposure, 31 (44%) students reported moderate prior exposure, 29 (41%) minimal exposure, 7 (10%) extensive exposure, and 4 (6%) no prior exposure. When asked whether AI tools should be formally incorporated into the curriculum, 43 (61%) students somewhat or strongly agreed, and 28 (39%) somewhat or strongly disagreed.

At baseline, 64 (90%) students reported having used AI tools for studying or schoolwork. Of those, 63 (98%) somewhat or strongly agreed that AI was positively impacting their learning, and 1 (2%) somewhat disagreed. Regarding clinical reasoning, 53 students (83%) somewhat or strongly agreed that AI was positively impacting their clinical reasoning, and 11 (17%) somewhat disagreed (Table 1).

AI Use During Studying

The average time spent studying was 185.6 (SD 89.7 minutes per diary over the 3-week period. The average time using AI tools was 35.8 (SD 35.8 minutes, representing 19% of overall study time.

Specific AI tools used over the 3-week period are reported in Table 2. The most commonly used tool was ChatGPT (63/71, 89%), followed by Google Gemini (22/71, 31%), AMBOSS AI (11/71, 16%), Notebook LM (6/71, 9%), Open Evidence (5/71, 7%), UpToDate AI (5/71, 7%), and DeepSeek (3/71, 4%).

Students reported using AI for a variety of purposes (Table 2). Passive uses were most common: simplified explanations of difficult concepts (53/71, 75%), learning about complex medical topics (51/71, 72%), answering practice questions (37/71, 52%), creating disease or treatment summaries (29/71, 41%), and summarizing lecture notes (26/71, 37%). Active uses included generating mnemonics (25/71, 35%), interpreting lab results or imaging findings (22/71, 31%), simulating patient interactions (19/71, 27%), creating study schedules (17/71, 24%), generating flowcharts or algorithms (9/71, 13%), and using spaced repetition techniques (7/71, 10%).

Diary Trends

Participants’ average study time and AI use were tracked over 7 diary days (Figure 3). Study time ranged from 159.0 minutes (diary 3) to 197.5 minutes (diary 1). AI use ranged from 29.1 minutes (diary 4) to 40.8 minutes (diary 6). The proportion of study time spent using AI ranged from 16% to 24% across diary days.

Among the 65 students who completed at least 1 diary entry, the most commonly reported passive uses of AI were simplified explanations of difficult concepts (n=50, 77%) and learning about complex concepts (n=49, 75%), followed by answering questions (n=29, 45%), class assignments (n=23, 35%), disease summaries (n=21, 32%), and summarizing notes (n=19, 29%). Active use was less common across all categories, with lab interpretation and simulating patients being the most frequently reported active uses (n=17 each, 26%), followed by mnemonics (n=16, 25%), study plans and clinical algorithms (n=12 each, 19%), and spaced repetition (n=8, 12%). The distribution of active and passive use types is shown in Figure 4.

‎

‎

Subgroup Analyses

There were no significant differences in AI use percentage by gender (P=.14), age (P=.16), and campus (P=.41). A significant difference was observed by year in training: third- and fourth-year students reported using AI during 31% (127.5 min) of their study time, compared to 16% (145.2 min) among first- and second-year students (P=.003). There were no significant differences in AI total minutes by gender (P=.39), age (P=.93), and campus (P=.17). A significant difference was observed by year in training: third- and fourth-year students reported using AI during 31% (132 min) of their study time, compared to 16% (155 mins) among first- and second-year students (P=.003). The Cohen d for year in school on percent AI study (d=0.97) indicates a large effect (Table 3).

Regression Analyses

Two multivariable linear regression models were conducted to examine predictors of AI use (Tables 4 and 5). In the first model predicting the percentage of study time spent using AI (n=60), clinical-year students reported a significantly higher proportion of study time using AI compared with preclinical students (adjusted β=−16.98, 95% CI −27.90 to −6.07; P=.003), representing a medium effect size (Cohen f²=0.18). Age, active use classification, and campus were not significant predictors (all P>.10), with negligible to small effect sizes. The full model explained 18% of the variance in AI study time proportion (R²=0.18, Cohen f²=0.23).

In the second model predicting total minutes of AI use (n=64), active use was the only significant predictor (adjusted β=108.42, 95% CI 23.11-193.74; P=.01), representing a medium effect size (Cohen f²=0.11), indicating that students classified as active users spent approximately 108 more minutes using AI over the study period compared with passive users. Year of training, age, and campus were not significant predictors (all P>.10). The full model explained 13% of the variance in total minutes of AI use (R²=0.13, Cohen f²=0.14).

Multilevel Models

Table 6 describes the multilevel model predicting percentage of study time spent using AI (n=248 diary entries from 59 students). In this model, preclinical students reported significantly lower AI use than clinical-phase students (β=−11.30, 95% CI −21.08 to −1.51; P=.02), consistent in direction and significance with our primary regression in Table 4. Active use, age group, and campus were not significant predictors (all P>.10). The intraclass correlation coefficient (ICC) was 0.47 (95% CI 0.34‐0.60), indicating that slightly under half of the total variance in percentage of AI use was attributable to stable between-student differences, while just over half reflected fluctuation within students across their own diary entries.

In the parallel model predicting AI minutes per diary entry (n=253 entries from 59 students; note this outcome is defined at the diary-entry level rather than as the person-level total used in Table 5), none of the fixed effects reached significance (all P>.10). The direction of the active-use effect was consistent with our original aggregated model, but the association did not remain significant when examined at the diary-entry level (β=10.89, 95% CI −4.68 to 26.46; P=.17). The ICC for this outcome was 0.55 (95% CI 0.43‐0.66).

An exploratory model additionally including diary wave (entry 1‐7) as a fixed effect found no significant linear trend in AI use across the 21-day period (all P>.05), suggesting day-to-day fluctuation rather than systematic drift over time.

To further characterize this variability, we calculated each student’s own standard deviation in percentage AI use across their diary entries. Among the 53 students with or more valid diary entries, the mean within-person SD was 10.6% (SD 8.2%; range 0%‐28.4%; Table S1 in Multimedia Appendix 2), indicating substantial day-to-day fluctuation within individual students that would not be visible using person-aggregated scores alone (Figure S1 in Multimedia Appendix 2).

Discussion

Principal Findings

This pilot study found that medical students used AI tools during 19% (35.8 min) of their total study time. Clinical-phase students (MS3-MS4) were more likely to use AI for studying than preclinical students (MS1-MS2), and students who engaged actively with AI spent more total time using these tools. Overall, AI use was primarily passive, with students relying on tools such as ChatGPT and Google Gemini to simplify concepts, answer practice questions, and generate summaries. AI use varied considerably from day to day within the same student, with roughly half of the differences we observed in AI use reflecting within-person fluctuation rather than stable differences between students. This suggests that a student’s AI use is less a fixed trait and more likely something that shifts with daily demands, such as upcoming exams, assignment deadlines, or workload.

Clinical-year students were more likely to use AI for practice questions, likely reflecting preparation for licensing exams such as Comprehensive Osteopathic Medical Achievement Test (COMAT) and USMLE Step 2 CK during the study period. These students may also face greater time pressure during rotations and use AI as an on-demand resource for rapid review. The higher proportional use among clinical students may also reflect greater familiarity with AI tools through clinical exposure and a curriculum that offers fewer structured study resources compared to didactic years.

Students classified as active AI users spent more total time using AI, likely due to iterative questioning, critical evaluation, and integration into study or clinical reasoning tasks. In contrast, passive use tends to involve brief, one-directional consultation [22]. When we looked more closely at individual diary entries rather than each student’s overall total, the data became more nuanced. Total AI minutes appeared higher among active users when we summed across their entire 3-week diary period. However, this advantage was not visible when comparing single diary entries to one another. Active users may have used AI more consistently over the study period and submitted more diary entries rather than spending significantly more time on AI in any single sitting. This is helpful for thinking through “active use” patterns. Active users are not necessarily more intensively engaged each time they sit down to study, but could instead engage with AI tools more frequently or more consistently over time.

Our data do not capture whether students verified or critically evaluated AI outputs. The predominance of passive use patterns in our sample raises questions about whether students are using critical thinking to safely use AI in clinical contexts. This is consistent with findings from other studies suggesting that many students do not routinely check AI outputs against authoritative sources [23]. This gap represents an important area for curriculum development and institutional guidance.

Comparison With Prior Work

Although there are some data that suggest that AI tools may improve efficiency and personalization, evidence on their impact on deep learning remains mixed. Some studies suggest AI use may hinder conceptual understanding, synthesis, and application of knowledge and raise concerns about the misinterpretation or acceptance of inaccurate outputs [5,22]. One study of accounting students found lower exam performance associated with ChatGPT use [5]. Given that most use in our study was passive, the implications for learning outcomes and cognitive offloading warrant further investigation.

To date, no prior studies have quantified AI use as a proportion of total study time. A 2023 US survey of 415 medical students found that 52% reported using ChatGPT for schoolwork. International studies of self-reports vary widely: 42% in Sudan, 62% in China, and nearly 90% in Palestine [6-8]. A global survey of 3839 college students found that 86% reported using AI for studying [24], suggesting continued growth in adoption.

In our study, 61% (n=231) of the students supported formal integration of AI into curricula. Early educational interventions show promise: the extracurricular AIM program improved AI literacy among premedical students [25], and a 14-week randomized trial of a ChatGPT-based coaching tool demonstrated gains in self-directed learning and critical thinking [26]. Other tools have shown more modest effects [27]. These findings suggest that carefully designed AI tools may enhance learning.

Limitations

This study has several limitations. Self-reported data, even with a diary methodology designed to reduce recall bias, remain subject to social desirability bias and time estimation error; self-reported minutes may not precisely reflect actual usage, and triangulation with device-level data was not possible.

A standardized, validated framework for classifying active versus passive AI use does not yet exist. Recent commentary similarly argues that this binary may be too coarse to capture the complexity of student engagement with AI [20], with related scholarship proposing more graduated constructs such as students’ epistemic agency [21]. Our classification should therefore be understood as a provisional, formative operationalization rather than a validated instrument.

Our nonrandom convenience sample from 2 osteopathic schools, combined with a 5.3% (n=71) response rate, limits generalizability. We cannot rule out self-selection toward students more engaged with or curious about AI, which would tend to inflate our estimates of AI adoption and usage intensity; the findings should be interpreted as characterizing a self-selected subgroup rather than the broader student population at either institution.

The modest sample size (n=71) reduced power for subgroup analyses, reflected in wide CIs. Supplementary mixed-effects models using diary-level data (Table 6) confirmed that within-person fluctuation accounted for roughly half of the total variance in AI use (ICC=0.47‐0.55) and corroborated our primary training-year finding, though the diary-level active-use association did not reach significance, likely reflecting reduced precision at this more granular level.

Finally, our 3-week study period (April-May) may not capture AI use patterns at other points in the academic calendar. We lacked a validated digital literacy measure and did not assess academic performance or learning outcomes, limiting the interpretation of our findings for curricular recommendations. Unmeasured confounders, such as AI proficiency and baseline academic performance, may also have influenced results.

Conclusions

Although these findings are preliminary and should be interpreted with caution, they suggest that medical students are incorporating AI into a substantial proportion of their study time. Distinct patterns by training stage and engagement style were observed in this pilot cohort. Because our study did not measure learning outcomes, we cannot draw conclusions about whether AI use is beneficial or harmful. Given mixed evidence regarding the impact of AI on deep learning and potential risks related to uncritical acceptance of AI-generated content, these findings highlight the need for further research on learning outcomes. Institutions may wish to consider providing guidance on responsible AI use, including critical evaluation, verification of outputs, and integration into evidence-based study strategies. Future studies should include larger, more diverse samples; validated digital literacy and outcome measures; and analytical designs that exploit the longitudinal structure of diary data. The digital diary approach used in this study offers a practical method for capturing real-time AI use and may be useful for evaluating future educational interventions.